Recovering Distributions from Gaussian RKHS Embeddings
نویسندگان
چکیده
Recent advances of kernel methods have yielded a framework for nonparametric statistical inference called RKHS embeddings, in which all probability distributions are represented as elements in a reproducing kernel Hilbert space, namely kernel means. In this paper, we consider the recovery of the information of a distribution from an estimate of the kernel mean, when a Gaussian kernel is used. To this end, we theoretically analyze the properties of a consistent estimator of a kernel mean, which is represented as a weighted sum of feature vectors. First, we prove that the weighted average of a function in a Besov space, whose weights and samples are given by the kernel mean estimator, converges to the expectation of the function. As corollaries, we show that the moments and the probability measures on intervals can be recovered from an estimate of the kernel mean. We also prove that a consistent estimator of the density of a distribution can be defined using a kernel mean estimator. This result confirms that we can in fact completely recover the information of distributions from RKHS embeddings.
منابع مشابه
Kernel Choice and Classifiability for RKHS Embeddings of Probability Distributions
Embeddings of probability measures into reproducing kernel Hilbert spaces have been proposed as a straightforward and practical means of representing and comparing probabilities. In particular, the distance between embeddings (the maximum mean discrepancy, or MMD) has several key advantages over many classical metrics on distributions, namely easy computability, fast convergence and low bias of...
متن کاملModelling transition dynamics in MDPs with RKHS embeddings
We propose a new, nonparametric approach to learning and representing transition dynamics in Markov decision processes (MDPs), which can be combined easily with dynamic programming methods for policy optimisation and value estimation. This approach makes use of a recently developed representation of conditional distributions as embeddings in a reproducing kernel Hilbert space (RKHS). Such repre...
متن کاملHypothesis testing using pairwise distances and associated kernels
We provide a unifying framework linking two classes of statistics used in two-sample and independence testing: on the one hand, the energy distances and distance covariances from the statistics literature; on the other, distances between embeddings of distributions to reproducing kernel Hilbert spaces (RKHS), as established in machine learning. The equivalence holds when energy distances are co...
متن کاملComparing Structure Learning Methods for RKHS Embeddings of Protein Structures
Non-parametric graphical models, embedded in reproducing kernel Hilbert spaces, provide a framework to model multi-modal and arbitrary multi-variate distributions, which are essential when modeling complex protein structures. Non-parametric belief propagation requires the structure of the graphical model to be known a priori. Currently there are nonparametric structure learning algorithms avail...
متن کاملDistribution Embeddings in Reproducing Kernel Hilbert Spaces
The “kernel trick” is well established as a means of constructing nonlinear algorithms from linear ones, by transferring the linear algorithms to a high dimensional feature space: specifically, a reproducing kernel Hilbert space (RKHS). Recently, it has become clear that a potentially more far reaching use of kernels is as a linear way of dealing with higher order statistics, by embedding proba...
متن کامل